Page-Level Testing with Cypress & cypress-axe
On this page
In the previous section on Cypress Component Testing, we looked at the cypress-axe extension to check for accessibility issues at the component level.
As mentioned, cypress-axe is more useful when used to test at the page level than at the component level.
However, the types of issues that it identifies should be caught earlier in the process.
The cypress-axe plugin is powered by axe-core, the same library that powers Storybook and the axe Accessibility Linter for VS Code.
That said, the goal of this lesson is to become more familiar with the Cypress UI and get used to thinking about testing on the page level.
Find Accessibility Issues on the Page-Level with cypress-axe
Let’s write a test that uses cypress-axe to ensure we don’t have any accessibility issues on the Home page.
As we saw in the lessons on Cypress Component Testing, we’ll tell Cypress to inject axe in the beforeEach block.
We’ll write a new it block with a description that we should not have any accessibility violations on load.
Like we did before, we’ll call cy.checkA11y() but this time we won’t pass any options to it since we want it to run all of the axe-core rules at the page level.
The test looks like this:
describe('HomePage', () => {
beforeEach(() => {
cy.visit('http://0.0.0.0:1234/')
cy.injectAxe()
})
it('should have no accessibility violations', () => {
cy.checkA11y()
})
})Running Cypress, the UI will show us the test results and the issues that are present on the page.
There are four issues that Cypress has identified:
- A missing button name.
- Missing image alt descriptions.
- A landmark issue.
- A couple of region issues.
It is important that these issues get resolved, as they negatively impact the experience for screen reader users.
Remember, we could catch the issues identified in this first test earlier in the process with VS Code or browser extensions. It’s still useful to include Axe in integration tests in case other devs on the team don’t have the other tools installed.
Video Transcript
let's look at a test. So our homepage test file is pretty basic right now. it says describe homepage. So it uses the same kind of API for writing tests as all these other flavors have so far describe statements before each it, and then a test you know, describing a scenario.
And so that's pretty cool. So we've got in our, before each we're saying side up visit and telling it to go to http local host port 1, 2 34. So that's why it was important when it was trying to start up another server. You know, maybe, you know, if you've got like a more robust set up, maybe you can make this more like listening for what start server and test made, or maybe you make it a specific port that like always opens Cypress.
And then if you have 1, 2, 3, 4 for your like manual testing and, you know, make that separate you, there's more options that you could do. This just happens to use the same one. So it wasn't working quite right until I killed off that other terminal process. We're gonna write some tests to assert that things are happening the way that we expect.
So like to run Cypress axe I could say it should have no accessibility violations. And this is going to run Cypress, like inject the axe-core library into this browser instance and run the accessibility API. So all of the rules that exist in the axe core framework, we saw some of them in storybook.
So far, if you're using the axe extension in your browser, it would be similar kind of, depending on what version of the extension you have and what version of axe core you're running as a package. But so with this, we first need to do side, dot inject axe in. I'm gonna do that in our, before each, so that it's available by the time that we go to run our it statement.
And within here, the command To run X is side check a11y and so that will inject axe will do that. Step of getting the X core library into the right spot in our browser. Not in the Cypress, you know, frame, but within the actual browser where it needs to run. So the right context and check.a11y or C dot check a11y sort of like doing the checking and the assertion in one command.
So that's it for the test. So if I hit save over here and come over to cy I'm gonna click on homepage test, cuz that's where I just added this code. So it's gonna open a browser instance directly to this test. It will go to that page. So we're rendering the whole page. Now we've got components all together, we're on our homepage and I can see that it failed on a number of items.
So it says four accessibility violations were detected expected four to equal zero. So that's why I say it kind of like it does the checking and the assertion in one in one command. And so for this, something that might not be immediately obvious when you're using Cypress axe is that the output is in the browser console.
So if I do on my Mac option command, I, I can get over here to my console. Oh, we have an issue with one of our props on our MegaNav. Let's come back to that a little bit later. But I can come through here in all these steps in the test body. If I click on any of these. I can come and learn more information.
So it does say printed output to your console. But you might not know that cuz it's kind of hidden behind a click and you have to interact with it. But this is where the details really go that are important to be able to use this tool. So there's a number of items here, like which error, you know, the impact level.
So for something like button name, that's a critical violation. It maps to the web content, accessibility guidelines name role value. We saw that when we got started today looking at WCAG there's other tags here. So WCAG 2A what CAG four 12 for that specific success criterion it also maps to section 5 0 8 for the United States federal government.
That's related to procurement for government and education type things. It's really anything that the government in the us spends money on. They have this section 5 0 8 standard that really references WCAG. So it's kind of like if you meet WCAG, the section 5 0 8 thing should be met as well.
It's also part of automated conformance testing and it's got kind of an ax core internal category of name, role value. So the tags are important because sometimes. If it says WCAG and a number it's actually tied to a WCAG success criterion, that is a requirement. What we call normative. It's something that, you know, in a court of law for you to say that something is compliant and accessible or conformant, as we say, you have to meet these ones.
These are called violations. Compare that to something like a best practice, which under tags, doesn't have all those details from section 5 0 8 or WCAG and you'll see this in the browser extension as well, but these details are important for prioritization. So if you have a bunch of issues and you can only tackle them so much in this sprint or this work period, maybe you tackle the violations first.
I'm thinking about user impact. It's a way that we can kind of rank what we fix first. Some of the best practices are best practices and you don't necessarily need to follow them. A lot of them are worth doing like these landmark rules. This one says, ensures all page content is contained by landmarks and we can come in here and see, well, what elements are they talking about?
It's pretty much everything. There's no landmarks on the page. So let's go fix that. But I wanted to point out these tags to you because I've seen teams kind of miss the fact that something is a best practice and they're like pulling their hair out with stress. Like, oh my gosh, we have so much to do.
How are we ever gonna prioritize stuff? The tags can help, you know, maybe something WCAG, you know, text alternatives, 1.1 0.1, that's a level one, or level a success criterion. So there's A, double A, triple A. And that's another lever that you can pull to kind of prioritize things like fix the level A's first, then the double A's, then some of the triple A's if those come up and then the best practices, or maybe you do level a double a and best practice, you know, you kinda like craft your policy.
But that stuff's relevant, even though we're looking at automation, we're still, you know, we can only do so much at once sometimes. So that stuff's important. So let's go fix some stuff, cuz we could make this page a lot more accessible and you know, we're using Cypress axe here. So if like this, if we're catching this, this late in the game, you know, it was Cypress integration testing, you know, you might be not catching everything now.
Like hopefully you have process where you're manually testing earlier cuz you know, I keep telling you how important that is. It's a little bit late to be catching stuff with acts at this point, but maybe you accidentally miss something or maybe, you know, axe is running when the page loads. So whatever it can find in its state at this local host port 1, 2 34, like that state, when it loaded, you could make Cypress go and operate modals and menus and run it again.
And maybe, maybe that automation, you know, because you can automate that. That's stuff that you're not having to test manually necessarily. You could catch some low hanging fruit with automation, getting the page in all these different states. So I think better late than never just know that like this, hopefully isn't the first time that you're catching some of this stuff.
Like, hopefully by now in this part of your development workflow, you're already catching things like image, alt text and landmarks and that sort of thing. But Hey, if you catch it now and fix it before it ships to production better than not catching it at all right. So let's go fix some stuff.
🛠 Challenge: Fix the Issues Identified by Cypress
Cypress has identified several issues that can be fixed with some refactoring of the CampSpots app markup. Work through the issues until the test passes successfully.
Selecting an item in the sidebar and the DevTools Console will display additional information from Cypress about the issues it has identified.
As seen in the screenshot below this including specific Nodes, rule tags such as WCAG success criteria or best practices, and a link to learn more about the identified issue.
🛠 Solution: Fixing Identified Issues
Updating Alt Descriptions on the Homepage
Using the Nodes output from the console can help locate where in the codebase the images are missing alt attributes. We can search by filename or for the classname from the target CSS selector.
Image alt descriptions can be quite subjective.
The “Tents at Sunset” image on the homepage contributes content to the page but lacks an alt attribute. For this one, add a description of the photo.
There’s also a missing alt attribute on the CampSpots logo image. This we can treat a bit differently.
An alt description of the image being “tent image” probably isn’t the most useful description to include. Because the image is wrapped in a link along with a text span that says the name of the site, a screen reader would say something along the lines of “Link: tent image CampSpots”.
However, if we don’t add an alt attribute, the screen reader will read out aloud the cryptic, cached filename from the image’s src.
For the logo image in header.js, using an empty string for the alt attribute will work fine since it has redundant content to its span sibling. This way the screen reader will announce “Link: CampSpots” for the wrapping link element and ignore that there’s even an image there.
The alt Decision Tree from W3C WAI is a super useful resource for coming up with descriptions for your images, including how to decide whether something is decorative!
Video Transcript
so we've got some accessibility violations on our homepage that is page home.
We have that routed up in our app JS file. We have a router in there that that's where homepage comes in at the, the route or slash of our URL. So that pulls in components page home. So that component let's open that up. So we've got, oh, well right there. I can see we're missing an, an alt attribute for an image.
So that would definitely fail. So that image coming back to our browser oh, it's this one. So this is helpful. So when I'm on image alt, it actually highlights what element on the page we're talking about, cuz it's not this image up in the hero area. That's a background image, but this one down here is missing alt text.
So, and I can also look at the nodes and see there's three of 'em there's this tense thing. There's also this camp spot's logo, I guess that one is highlighted and then all the way down at the bottom, we have this little footer logo. So one in the header, one in the footer, one in the body of this homepage.
The header and the footer logo then will be fixed for any page that pulls those in. So that's nice. We fix it here. And then any other pages will benefit from that change. So let's this alt text for this tense image. I'm gonna give it some fancy alt text of a festival of tents at sunset.
How nice is that? I feel like Bob Ross so we've got that one was on the page. We've also got our header header component. That's repeated on every page. Here's the CampSpots logo. It's part of an anchor. So there's a link wrapping this image as well as a span. So I'm gonna make a slightly different judgment call for the alt text attribute or the alt attribute on this image.
I'm gonna make the executive decision to make this alt with an empty string, because this is part of an anchor it's accessible name. As we call it will be derived from this span that says camp spots. So the logo image, if I added camp spots as the alt text, it would be redundant and it would make this link say camp spots, camp spots, which we just don't really need.
So with an empty alt, it will skip by this image because we didn't really need it. It's redundant with this text. So this anchor now it's got at least one word populated, populating its successful name. And we won't have the issue where this file name is read because that's what happens when you don't have an alt attribute.
It reads the file name, which in most modern file systems, you get these long cached file names, which are a nightmare to have read aloud.
Refactoring the Header & Footer to use Landmarks
One of the issues that Cypress found was a missing image alt attribute in the footer, which currently has its markup inside of App.js.
Adding an alt description of “CampSpots” to the footer image is a quick fix.
While we’re working with the footer, another thing Cypress found was that the current markup doesn’t use landmark elements. Landmarks are a best practice for screen reader navigation and are worth including on pages.
To fix this, we can swap out the footer’s <div> with the semantic <footer> landmark element.
The Footer markup now looks like this— feel free to extract it to its own component!
// inside of App.js
<footer id="footer">
<div className="layout">
<div id="footer-logo">
<img src={imgFooterLogo} alt="CampSpots" />
</div>
</div>
</footer>We also need to fix the Header component to use the <header> landmark.
Inside of components/header.js swap out the <div> with <header>.
Finally, we add our only main in our App.js by switching the <div> to <main>. Leaving us with three top level landmarks.
Video Transcript
I can also tell right here. I mean, this is divs mostly and header. There happens to be a top level landmark. As we learned in our semantic HTML & ARIA workshop, we could be a lot more semantic with this.
You know, and if we had already fixed this, that we wouldn't be having this issue in Cypress acts, but let's fix it while we're in here. So this div ID header, I'm gonna change to a header element. So because this is a top level landmark it will become a banner header. So there's only one of these.
You can only have one header banner, just like you can only have one main, let's go add main. So that is in our app dot JS. This is where our header component was pulled in. So every page will benefit from this header landmark. We will also get this main landmark. I'm gonna leave these IDs in here because if we had skip links or something that we're referencing these IDs or our style sheet, I can swap out this tag name and not have the can of worms of having to go change all those things.
We also have a div idea of footer that could be a footer content info landmark. So we have three top level landmarks. Now all of our content should be wrapped. I'm gonna come back to the Chrome browser and I'm gonna hit this run test button again. And we took it down to three . Oh, we, oh, we need one more alt text here, down in the footer.
This one we could say camp spots maybe could be, you know, it could be, maybe it needs some more thinking around what would make that image camp spots, image more unique. It's probably fine for now better than having the file name read. So I'm gonna come back over here
and then region, let's see what that one's about. So saying we have some content that is not contained by landmarks.
And this one I'm going to talk through a little bit more. So we've got a, a heading treatment on our site. So we've got individual pages that pull in our header and for our heading structure to get an H1 kind of up above our header, which has it has headings in it like H2s for each of these menu items.
You really want your page to start with an H1. So you have this dilemma of like, how do I get a, heading an H1 heading to come before these other headings? When my page content that's like, I'm like further down in the DOM tree. You just, you end up with this like heading order dilemma. And so to get around that I've been using something called a react portal.
And so this portal route is a way in a page component for me to go and inject an H1 higher up in the page with an H1 that's specific to that page. So like the about page or the events page, you know, if I want an H1 to get the right heading order, I'm putting it up higher in the page, but it created this scenario where like, we fixed one thing for the heading order.
And then now it's not contained by a landmark , but we already have a header, a banner header. So this technically is a best practice. So it's saying that all your content should be contained by landmarks. This H1 not being in a landmark. I don't, I personally like my professional opinion. There's like a bigger issue with our heading order and components here that maybe I could think through some more, but for an H1 heading to not be in a landmark, I don't actually know that that has much of an accessibility impact, so I can make this choice, you know, after really thinking through what's the user impact of this result.
It is best practice. If I have an H1 that's outside of landmarks, I personally don't really think it's a problem. Because some screen reader, users like many aren't gonna navigate by landmarks anyway, which is why these are best practice rules. They're kind of suggestions. Like if you follow them sure they can help.
But it's not necessarily wrong. to have an H1 that's not contained by landmark. It's the first thing in the page you know, ideally it should be part of the header landmark, and maybe there's a way I could get the portal route to go inside of the header landmark. I don't know. It's like we end up, you fix one problem, then you have another problem.
So we kind of have to always evaluate what's the user impact and having a good, heading structure that communicates the overall page hierarchy of all the content I'm gonna say is more important than making sure that this H1 is in a landmark, my own personal take. So you might get results. That are not, I don't know.
You might just sort of punt on it, like, okay, we're gonna leave that one or we can go ignore that one. Hopefully you can like disable the result somehow. Like maybe, you know how I configure Cypress acts. I could go exclude, like set it up, configure it. So I can specifically exclude that result for that element.
Axe core. The API does have that capability to configure it that way. Cuz what you wanna avoid is having things that your dev team is just trained to overlook. We're like, oh, don't worry about that. Cuz then it kind of waters down the utility of your tool cuz you're ignoring, you're like picking and choosing what to pay attention to.
So my next step, you know, if I'm deciding like, oh that's not really a problem. Like for this reason we've tested it with users. We know it's fine. The tool is making a suggestion. We are making an educated decision to deviate, try and configure the tool and comment like why that decision was made so that it won't keep coming up.
But then yeah, if you have it commented and documented, then you know, you have a paper trail
Add a Button Name to the IconButton Component
There’s one error left in Cypress and it has to do with the missing name for the IconButton component.
As it happens, we had fixed this issue back in the section on Unit Testing with Jest. But we didn’t update the actual usage in the application!
Inside of components/search-form.js, add a name prop with a value of “Submit” to the IconButton component:
<IconButton name="Search" onClick={() => console.log('Submit my stuff!')} />A Note About the H1 Region Error
CampSpots has an H1 up above the Header component which has H2s for all of the MegaNav buttons. We built this heading structure with a React Portal in the Semantics & ARIA Workshop.
We are seeing this error in Cypress because the H1 is not inside of a landmark, but in my opinion having a proper heading structure is more important.
Screen reader users are more likely to navigate a page by heading structure than they are by landmarks.
Ideally the H1 should be part of the header landmark, but it would require some rearchitecting to make it work.
I don’t agree with this being a violation, but on a team in the real world you shouldn’t build a bad habit by training them to overlook accessibility violations because “it’s okay” in certain situations.
Exclude an Element from cypress-axe
To exclude the header portal from being tested by cypress-axe, we can add a config object to the cy.checkA11y() call.
The object takes a key called exclude whose value is an array of selectors to ignore. In this case, we can use the target element from the console output that points to the portal.
Here’s what our test looks like now:
it('should have no accessibility violations', () => {
cy.checkA11y({
exclude: ['div#portal-root']
})
})Video Transcript
so now we have one issue where we still have a button that's missing a button name, so an icon button that isn't getting an ARIA label or, you know, off screen text or visually hidden text.
This button is not, not gonna work in a screen reader. So if I inspect that here, this is another kind of benefit of Cypress in the browser is I can come in and go learn more about it. So there is a button here. I do have to kind of like get in here with Cypress. It can be a little bit difficult. So right now I'm kind of getting thrown off by the Cypress highlight itself.
Whereas sometimes I can get around that by like, okay, stop showing me these highlights in Cypress. You're polluting my markup. I can come in here and click on maybe the, before each or like get earlier into the test body or test process to just get the Cypress stuff out of the way. Then I can come with this little picker and inspect this button in the Dom without all the Cypress stuff.
So this button submit it's a button element, so that's cool. But it just has this span in it. So this icon button instance. Is missing some sort of a name, an accessible name. So that helps me to just kind of understand what's going on here. Like in context, before I go back to my code to change something, what I like about Cypress is I've got a real browser environment.
I can go inspect it and kind of go, well, what is happening here? Because yeah, we wanna make sure we're actually, we understand the problem before we go and write code to fix it. So button name, that'll fix that one
let's fix that button name to get the satisfaction of fixing that. So that is within search form. Search form has an icon button and icon button. We've seen that before, so that pulls in aria label.
So it's got an aria label prop. If we give it a name, it should land in the right spot. So let's come back to search form and I will say icon button name search. Cool. So if I hit run all again, Cool. Now we just have that one region issue that hopefully we could configure and leave alone. You know, maybe we come back, try and revisit it, see if we can architect our app a little bit differently so that this, we can include it in a landmark.
But for now, I'm gonna choose to leave that one alone. So that's how you can use Cypress axe to kind of like go in, make sure you fix stuff. I could run, you know, come in here with Cypress, as I was saying and, and open the menu or maybe we come over to the passes page and we could write some tests for testing in this modal or something like that.
You have to actually get Cypress into that state, but then I could run Cypress axe again and kind of repeat the process of fixing stuff that way. It's pretty cool. Nice interactive way of doing things.
With all the issues from cypress-axe addressed, we have a passing HomePage test suite!
Write Cypress Axe Integration Tests for the Other Pages
Following the structure we used for testing the home page, add integration tests for the About and Listings pages.
Write an assertion for another element on the Home Page
With the Header test as a guide, write another assertion for an element on the Home page.



